1. Technical Field
The present invention relates to a range-finding system including a stereo camera system and a vehicle mounting the range-finding system.
2. Related Art
Collision avoidance systems involving the use of in-vehicle stereo camera systems have become more common. A stereoscopic image of the area in front of the vehicle is generated using the stereo camera systems, and an obstacle is detected and a distance to the obstacle is measured based on the generated stereoscopic image. The driver can then be alerted to take corrective action to avoid a collision or maintain a safe minimum distance between vehicles. Alternatively, the system can engage a control device such as the brakes and the steering.
FIG. 10 is a diagram for explaining a principle of range-finding in which two cameras are disposed in parallel. A camera 100a with a focal length f, an optical center O0 and an imaging plane S0 is disposed such that a direction of an optical axis extends in a upper direction in FIG. 10, and a camera 100b with the same focal length f is deposed in parallel with and spaced by a distance B with respect to the camera 100a on the right side.
An image of a subject A which is located at a distance d from an optical center O0 of the camera 100a in the direction of the optical axis forms an image at a point P0 which is the intersection of a straight line A-O0 and an imaging plane S0. With respect to the camera 100b, the subject A forms an image at a point P1 on an imaging plane S1. Here, an intersection of a line and the imaging plane S1, which line passes through an optical center O1 of the camera 100b and is parallel to the line A-O0, is indicated by P′0, and the distance between the point P′0 and the point P1 is indicated by p. The point P′0 is the same as the point P0 on the camera 100a. The distance p indicates a deviation amount P′0-P1 of the points on the images captured by these two cameras 100a and 100b and is called parallax. Then, a triangle A0-O0-O1 and a triangle O1-P′0-P1 are geometrically similar and thus the following formula I is given.
                    d        =                  Bf          p                                    (        1        )            where B represents the base line length of the triangle. If the base line length B and the focal length f are known, the distance d can be determined based on the parallax p.
In order for the above-described principle of range-finding by stereo camera system to hold, the two cameras 100a and 100b must be disposed precisely, as shown in FIG. 10. However, when the two cameras are mounted in a vehicle, it is difficult to dispose the cameras in parallel without some error, and therefore parallax correction techniques using signal processing are proposed.
As one example, after the stereo camera system is assembled, by capturing a test chart set at a known position, relative positions of the two cameras are accurately measured. Then, using the measured data and deforming the captured image, a pseudo-parallel arrangement can be realized. At this time, the image data is modified by affine conversion.
In another example, in order to correct non-linear deviation, such as aberration of distortion of the optical lenses, which cannot be modified only by affine conversion, a table is used the contents of which are determined based on data of a captured test chart set at a known position.
In yet another example, a technique to correct the deviation caused by distortion and temperature change in actual use is proposed. During actual use, it is difficult to capture the known test chart, unlike during the manufacturing process. Instead, using only how parallel the dividing line on the road is, the deviation amount is calculated.
Herein, to maintain consistently accurate range-finding in the above-described range-finding device, precise calibration is needed. Accordingly, various calibration techniques are proposed.
Known calibration techniques fall roughly into two categories: Those having set times for executing calibration and those having a degree of flexibility in modifying the image.
More specifically, the timing for executing calibration can be divided into two types: Those done during manufacturing and those done while in use. For example, in the calibration in manufacturing, the optical distortion of the lens and deviations arising in assembling are measured and corrected in manufacturing. By contrast, in the calibration in use, the influenced by temperature change and vibration over time is measured and corrected. The flexibility of the image deformation means, for example, affine conversion, non-linear conversion, and parallel movement having a predetermine offset.
However, in the calibration in use, a known object such as a test chart cannot be captured, and image conversion other than with a particular offset cannot be executed. Therefore, it is difficult to measure image deviation in the lateral direction accurately. However, when two cameras are set at ideal position, such as without aberration in assembling, corresponding points between a left image captured by a left camera and a right image captured by a right camera always show parallax in the lateral direction. Therefore, if a vertical component of the position difference between the corresponding points is present, it can be easily assumed that there is image deviation in the vertical direction based on the position difference between the corresponding points. By contrast, with the parallax in the lateral direction, a true value thereof depends on the distance to the object, and therefore detecting or estimating the amount of the deviation in the lateral direction is difficult.
In yet another example, using how parallel is the dividing line and the number of motionless objects, the parallax offset, that is, a parallel moving amount of the entire image in the left image and the right image in the lateral direction is estimated. However, image deviation in the lateral direction of the stereo camera system having two cameras provided in parallel is not limited to the parallel movement. Accordingly, the image deviation in the lateral direction cannot be measured and corrected without using the known test chart in use.
The types of lateral deviation possible in a stereo camera system are described below. The lateral deviation generated in the stereo camera system includes lateral deviation caused by rotation around a vertical axis and horizontal deviation caused by change in distortion characteristics. When a line of vision of the camera is slightly changed in a given direction, the lateral deviation caused by rotation around a vertical axis is moved to an opposite direction to the predetermined captured image. However, at this time, the image is not moved strictly in parallel.
For example, as illustrated in FIG. 11A, when a camera 100 is tilted to the left, a rectangle positioned in front of the camera is deformed as shown in FIG. 11B. Although FIG. 11B exaggerates the effect, when the camera 100 is tilted to the left, a rectangle positioned in front of the camera is deformed from the rectangle 110 to a trapezoid 120. In this case, the entire image is deviated to the right, the size of the right side image is reduced, and the left side of the image approaches the center position. Therefore, compared to a position B adjacent to the center position of the image, a left side point A is greatly moved to the right side. The right side of the image is expanded and the moved far from the center position, and a right end C is greatly moved to the right side.
As a result, the image deviation to right side is not uniform over the entire image, as illustrated in FIG. 12. In a comparative example, if the angle of deviation is very small, the trapezoid 120 shown in FIG. 11B is very near to the rectangle 110, which is regarded as pseudo-parallel movement. However, in order to detect the parallax more accurately, the parallax should be corrected not just regarding simple parallel movement but also as to deviation that varies depending on the pixel position.
Next, the lateral deviation caused by the change in the distortion characteristics is described. The focus lens system generally has aberration distortion characteristics. In an initial state, using the above-described table, non-linear deviation can be corrected but the distortion characteristics may change over time. In this case, the change in the center area of the image is small, but at the edges of the image the image deviation may be something other than parallel movement, such as expansion or contraction.
Furthermore, if deviation other than that which is symmetrical around the optical axis occurs, such as, tilt or eccentricity of the lenses, anisotropic distortion is generated, resulting in even more complicated deviation.